Closes #621

Proposed Changes

Handles data with missing values when querying to data frames. The query functions query_data_frame... have new optional parameter use_extension_dtypes.

def query_data_frame(self, query: str, org=None, data_frame_index: List[str] = None, params: dict = None,
                     use_extension_dtypes: bool = False):
    ...

def query_data_frame_stream(self, query: str, org=None, data_frame_index: List[str] = None, params: dict = None,
                     use_extension_dtypes: bool = False):
    ...

• when set to True, missing values are represented as pandas.NA and dtype of columns containing <NA> is of corresponding nullable extension dtypes from pandas package (ie. Int64, Float64,Boolean etc). Missing value can be checked using pandas.isna() function.
• when False (default), missing values are represented as None, and dtype of columns with missing values is either 'object' or 'float64' when type of values is numeric. This is a standard conversion behavior of data frames, see
  • https://note.nkmk.me/en/python-pandas-nan-none-na/
  • https://pandas.pydata.org/docs/user_guide/missing_data.html

Example output (with data from #621):

use_extension_dtypes=True

<bound method NDFrame.head of     result  table                    _start                     _stop                            _time _measurement  test_double  test_long
0  _result      0 2023-12-15 13:19:54+00:00 2023-12-15 13:19:57+00:00 2023-12-15 13:19:55.372000+00:00         test          4.0       <NA>
1  _result      0 2023-12-15 13:19:54+00:00 2023-12-15 13:19:57+00:00        2023-12-15 13:19:56+00:00         test         <NA>          1>

use_extension_dtypes=False

<bound method NDFrame.head of     result  table                    _start                     _stop                            _time _measurement  test_double  test_long
0  _result      0 2023-12-15 13:19:54+00:00 2023-12-15 13:19:57+00:00 2023-12-15 13:19:55.372000+00:00         test          4.0        NaN
1  _result      0 2023-12-15 13:19:54+00:00 2023-12-15 13:19:57+00:00        2023-12-15 13:19:56+00:00         test          NaN        1.0>

Note: the conversion of numeric values to extension dtypes works properly with pandas>=2.0, so in Python 3.7 environment, where the latest available pandas is 1.3.5, dtype of columns with NA values is 'object' ie. same as without the use extension types. For Python 3.8+, pandas 2.x is available.

Checklist

• CHANGELOG.md updated
• Rebased/mergeable
• A test has been added if appropriate
• pytest tests completes successfully
• Commit messages are conventional